Adversarial Illusions in Multi-Modal Embeddings

Multi-modal embeddings encode images, sounds, texts, videos, etc. into a single embedding space, aligning representations across modalities (e.g., associate an image of a dog with a barking sound). We show that multi-modal embeddings can be vulnerable to an attack we call "adversarial illusions." Given an image or a sound, an adversary can perturb it so as to make its embedding close to an arbitrary, adversary-chosen input in another modality. This enables the adversary to align any image and any sound with any text. Adversarial illusions exploit proximity in the embedding space and are thus agnostic to downstream tasks. Using ImageBind embeddings, we demonstrate how adversarially aligned inputs, generated without knowledge of specific downstream tasks, mislead image generation, text generation, and zero-shot classification

(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs

We demonstrate how images and sounds can be used for indirect prompt and instruction injection in multi-modal LLMs. An attacker generates an adversarial perturbation corresponding to the prompt and blends it into an image or audio recording. When the user asks the (unmodified, benign) model about the perturbed image or audio, the perturbation steers the model to output the attacker-chosen text and/or make the subsequent dialog follow the attacker's instruction. We illustrate this attack with several proof-of-concept examples targeting LLaVa and PandaGPT

Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems

We propose Februus; a new idea to neutralize highly potent and insidious Trojan attacks on Deep Neural Network (DNN) systems at run-time. In Trojan attacks, an adversary activates a backdoor crafted in a deep neural network model using a secret trigger, a Trojan, applied to any input to alter the model's decision to a target prediction---a target determined by and only known to the attacker. Februus sanitizes the incoming input by surgically removing the potential trigger artifacts and restoring the input for the classification task. Februus enables effective Trojan mitigation by sanitizing inputs with no loss of performance for sanitized inputs, Trojaned or benign. Our extensive evaluations on multiple infected models based on four popular datasets across three contrasting vision applications and trigger types demonstrate the high efficacy of Februus. We dramatically reduced attack success rates from 100% to near 0% for all cases (achieving 0% on multiple cases) and evaluated the generalizability of Februus to defend against complex adaptive attacks; notably, we realized the first defense against the advanced partial Trojan attack. To the best of our knowledge, Februus is the first backdoor defense method for operation at run-time capable of sanitizing Trojaned inputs without requiring anomaly detection methods, model retraining or costly labeled data.Comment: 16 pages, to appear in the 36th Annual Computer Security Applications Conference (ACSAC 2020

GROWTH on S190425z: Searching Thousands of Square Degrees to Identify an Optical or Infrared Counterpart to a Binary Neutron Star Merger with the Zwicky Transient Facility and Palomar Gattini-IR

The third observing run by LVC has brought the discovery of many compact binary coalescences. Following the detection of the first binary neutron star merger in this run (LIGO/Virgo S190425z), we performed a dedicated follow-up campaign with the Zwicky Transient Facility (ZTF) and Palomar Gattini-IR telescopes. The initial skymap of this single-detector gravitational wave (GW) trigger spanned most of the sky observable from Palomar Observatory. Covering 8000 deg2 of the initial skymap over the next two nights, corresponding to 46% integrated probability, ZTF system achieved a depth of ≈21 m AB in g- and r-bands. Palomar Gattini-IR covered 2200 square degrees in J-band to a depth of 15.5 mag, including 32% integrated probability based on the initial skymap. The revised skymap issued the following day reduced these numbers to 21% for the ZTF and 19% for Palomar Gattini-IR. We narrowed 338,646 ZTF transient "alerts" over the first two nights of observations to 15 candidate counterparts. Two candidates, ZTF19aarykkb and ZTF19aarzaod, were particularly compelling given that their location, distance, and age were consistent with the GW event, and their early optical light curves were photometrically consistent with that of kilonovae. These two candidates were spectroscopically classified as young core-collapse supernovae. The remaining candidates were ruled out as supernovae. Palomar Gattini-IR did not identify any viable candidates with multiple detections only after merger time. We demonstrate that even with single-detector GW events localized to thousands of square degrees, systematic kilonova discovery is feasible

GROWTH on GW190425: Searching thousands of square degrees to identify an optical or infrared counterpart to a binary neutron star merger with the Zwicky Transient Facility and Palomar Gattini IR

The beginning of the third observing run by the network of gravitational-wave detectors has brought the discovery of many compact binary coalescences. Prompted by the detection of the first binary neutron star merger in this run (GW190425 / LIGO/Virgo S190425z), we performed a dedicated follow-up campaign with the Zwicky Transient Facility (ZTF) and Palomar Gattini-IR telescopes. As it was a single gravitational-wave detector discovery, the initial skymap spanned most of the sky observable from Palomar Observatory, the site of both instruments. Covering 8000 deg$^2$ of the inner 99\% of the initial skymap over the next two nights, corresponding to an integrated probability of 46\%, the ZTF system achieved a depth of $\approx$\,21 $m_\textrm{AB}$ in $g$- and $r$-bands. Palomar Gattini-IR covered a total of 2200 square degrees in $J$-band to a depth of 15.5\,mag, including 32\% of the integrated probability based on the initial sky map. However, the revised skymap issued the following day reduced these numbers to 21\% for the Zwicky Transient Facility and 19\% for Palomar Gattini-IR. Out of the 338,646 ZTF transient "alerts" over the first two nights of observations, we narrowed this list to 15 candidate counterparts. Two candidates, ZTF19aarykkb and ZTF19aarzaod were particularly compelling given that their location, distance, and age were consistent with the gravitational-wave event, and their early optical lightcurves were photometrically consistent with that of kilonovae. These two candidates were spectroscopically classified as young core-collapse supernovae. The remaining candidates were photometrically or spectroscopically ruled-out as supernovae. Palomar Gattini-IR identified one fast evolving infrared transient after the merger, PGIR19bn, which was later spectroscopically classified as an M-dwarf flare. [abridged